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ABSTRACT 



This two-week pilot study addressed one of the DoD’s most difficult 
training problems: how to maintain “use or lose” language skills among its 
linguists. The study, which involved over 300 linguists at 61 installations 
worldwide, was designed to support the Air Force strategy of “Global 
Engagement” by regularly providing pedagogically sound language 
maintenance materials in small doses, and encouraging linguists to 
maintain skills by building an interactive, supportive community of 
professionals. Participants were enrolled in two email distribution lists: an 
Announce list, which pushed a Russian “vitamin” lesson to linguists each 
day for ten consecutive business days, and the Discuss list, where 
linguists were encouraged to comment on lessons, discuss relevant 
issues, and build on-going relationships with other participants. Using 
existing technology, the study demonstrated the feasibility of the lesson 
delivery system and the potential of networking linguists into an online 
community via a listserv. Initial recruitment of participants was 
problematic due to the lack of an Air Force-wide database of linguist. 
Those that did participate were highly enthusiastic throughout the project. 
The all-inclusive, concise lesson format was found to be user-friendly and 
extremely useful for language maintenance. The Discuss list proved to 
be a useful forum for community building as linguists took advantage of 
the opportunity to express views and opinions on issues relevant to the 
group and to receive feedback. Of those subjects that completed the end 
of study survey, 96% indicated they agreed or strongly agreed this was 
an effective method of language maintenance. The article closes with 
recommendations on developing and conducting distance-learning 
courses of this type for military audiences. 



Introduction 



Distance Learning (DL), defined as “any formal approach to 
learning in which a majority of the instruction occurs while the educator 
and learner are at a distance from one another,” (Verduin & Clark, 1 991 ) 
has grown explosively in the past several years. While DL has long been 
conducted using paper-based correspondence courses, tele-courses, and 
other means, the advent of the Internet and it’s widespread usage has 
made it possible to reach virtually any audience anywhere in near real- 
time. According to a recent Gallup Poll, 54% of Americans reported using 
the Internet from home, school, or work on a regular basis. Of those, fully 
90% use the Internet for email, while 95% use it to find information. 
(Gallup Poll, 2000) Clearly, access to the technology for Internet or 
email-based DL is now widespread. Academia and industry have taken 
note and are developing training materials to be delivered to these new 
markets at break-neck speed. 

In the armed forces, the ever-growing complexity of technology 
requires longer and more difficult training. Exacerbated by dwindling 
resources, the increased training demands have forced decision makers 
to look for better and more cost efficient methods to maintain required 
skill levels. Diminished resources along with increased operations tempo 
limit the time that critical personnel can be removed from their 
assignments to obtain necessary training. Consequently, the possibilities 
of saving time and money via DL have become more important to the 
military community as well. 

This is the second of two studies undertaken by the authors in an 
attempt to discover the best uses of newly available technology to solve 
one of DoD’s most difficult training problems-how to develop and 
maintain language skills. These skills are needed to support the Air 
Force mission of “Global Engagement” which mandates 10% of Air Force 
officers be proficient at the 2/2 skill level (reading/listening) in a foreign 
language. (Air Force Foreign Area Officer Program Home Page, 1999) 
(See appendix A for an explanation of the 0-3 proficiency rating scale as 
tested using the Defense Language Proficiency Test-DLPT). 

In the previous DL study conducted with a limited group of 
Russian military linguists, the authors found that technology-enhanced 
language training would be more effective if language maintenance and 
language development were conducted separately. Initial development of 
language skills requires a significantly larger investment of time and 
energy by the learner than the maintenance of skills already acquired. 
(Supinski, Sutherland, & Valentine, 1999) The current study focuses on 
the latter issue-language maintenance-conducted by pushing via email 
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short, self-contained pedagogically-sound lessons to linguists on a daily 
basis. The intent was to facilitate maintenance by providing regular 
exposure to authentic materials in manageable packages. 

This study also addressed motivation to adhere to a maintenance 
effort. Motivation to continue with a DL program is improved if linguists 
have opportunities and activities to foster collaboration and socialization 
thereby becoming a vibrant online community of likeminded learners. 
(Champagne & Wisher, 2000; Oxford, Park-Oh, Ito, & Sumrall, 1993; 
Preece, 2000) Consequently, this study used a listserv to establish an 
online learning community that would provide an effective and motivating 
learning environment. 

Background 

Since President Clinton issued Executive Order 1311 in 1999, 
which mandated that the Department of Defense (DoD) find ways to use 
technology to improve training and educational opportunities for Federal 
employees in collaboration with academia and industry, Washington has 
been active in pursuing the application of technological advances for 
educational purposes (Clinton, 1999). In July 2000, The President’s Task 
Force on Federal Training Technology responded to the mandate with the 
following statement: ‘The Federal Government will need to evolve with 
the Internet age and move aggressively toward using learning technology 
to provide ‘anytime’ and ‘anyplace’ training opportunities to its workforce.” 
(President's Task Force on Federal Training Technology, 2000) 

Similarly, the US Congress formed the bipartisan Web-based 
Education Commission, which delivered its report in December 2000: 

“The Power of the Internet for Learning: Moving from Promise to 
Practice.” Chaired by Senator Bob Kerrey, this Commission issued a call 
for action to make powerful new Internet resources, especially broadband 
access, widely and equitably available for all learners; provide continuous 
and relevant training and support for educators and administrators at all 
levels; build a new research framework of how people learn in the Internet 
Age; develop high quality online educational content that meets the 
highest standards of educational excellence....” (US Congress Web- 
based Education Commission, 2000) 

Due to its cost-effectiveness, the Commission expects a shift to 
the use of web-based training, estimating dramatic growth in the e- 
learning market from its current level of $3.6 billion to nearly $25 billion by 
2003. Furthermore, the report contains information on a major initiative of 
the U.S. Army to educate soldiers through a $600 million effort called the 
Army University Access Online. By issuing laptops to provide soldiers 
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with access to DL opportunities, Secretary of the Army Louis Caldera 
believes that this program will help recruiting, will improve retention, and 
will help produce better educated soldiers prepared for the missions and 
battlefields of tomorrow. (US Congress Web-based Education 
Commission, 2000) 

With the rush to make courses available online, many ineffective 
materials and courses have been generated. In fact, Wired magazine 
reports students who sign up for online courses seldom graduate, with 
some online courses suffering dropout rates as high as 80 percent. 

(Delio, 2001) However, the following three examples-one from academia 
and two business enterprises-demonstrate plainly that online courses 
can be successful when constructed thoughtfully using sound 
pedagogical principles. 

Englishtown. An outstanding example of online foreign language 
instruction with very low attrition rates is Englishtown with approximately 
90 percent of students completing its courses. (Littell, 2001 ; Vande 
Vrede, 2001) Englishtown is reportedly the world's largest online 
language school and offers 100% web-based English language training to 
individuals and companies worldwide. Using a combination of self-study 
and synchronous, teacher-led discussion, students enjoy great flexibility 
in their English language training. 

The website, www.englishtown.com . is both an online school and 
a virtual community for students of English around the world. From the 
homepage, students can complete courseware or visit the community 
section, which includes real-time chat rooms pertaining to a variety of 
topics. A Pen Pal service offers members the opportunity to correspond 
in English, meet new friends and practice writing skills. 

Parlo. A second effective online business enterprise is www.parlo.com , 
which delivers very short daily email lessons to subscribers for each of 
five languages. Their lessons are similar to the “vitamin” described in this 
paper, but much shorter and at a significantly lower skill level. Although 
there is little empirical data on retention due to the site’s newness, it was 
recently named one of the Best 100 Sites for 2001 by Yahoo Internet Life 
online magazine. (Parlo, 2001) 

GOLDEN. An example from academia of successful online instruction 
without attrition is offered by the German Online: Distance Education 
Network (GOLDEN) located at http://golden.unl.edu . Students enrolled in 
GOLDEN can earn graduate course credit and pursue and MA online 
through the University of Nebraska-Lincoln. Remarkably, the GOLDEN 
program has never lost a student. The Co-Director of GOLDEN, Dr. AN 

3 




10 



Moeller attributes the success in retention to several factors. Chickering 
and Ehrman’s seven guidelines for outstanding undergraduate teaching 
and effective distance learning (to be discussed later in this section) were 
used as a basis/context for designing the courses. Moreover, online 
participants are able to respond without the interruption of their more 
vociferous colleagues and have more time to reflect and organize their 
ideas, which can result in deeper learning. (Colomb & Simutis, 1996) 
Students can participate on demand and most importantly the responses 
generated are available in perpetuity. Students can review what was 
said, evaluate everyone’s comments, synthesize ideas, and then 
respond. (Bailey & Cotlar, 1994) Finally, Moeller (2001) believes that 
student responses evidence much deeper learning in terms of Bloom’s 
Cognitive Taxonomy. (Bloom & Krathwohl, 1956) 

An online community also plays a key role in the success of 
GOLDEN. According to Moeller (2001), the nature of the "community" is 
largely formed by the feedback provided the participants. In the 
“conference room,” participants read one another's responses and the 
instructors ask questions that push the participants to dig deeper, to 
probe further into thinking about teaching and learning. Furthermore, the 
participants work in groups to contribute to problem solving tasks online. 
Finally, participants frequently take multiple courses together, getting to 
know one another. They also make it a point to meet face-to-face at 
professional conferences (e.g., the American Council on the Teaching of 
Foreign Languages). (Moeller, 2001) 

Seven Principles 

To establish a basis for bringing technology and DL to higher 
education, Chickering and Ehrmann described the most cost-effective 
and appropriate ways to use computers and telecommunication 
technology to advance the “Seven Principles for Good Practice in 
Undergraduate Education” as follows: 

1 . Good Practice Encourages Contacts Between Students and Faculty 

2. Good Practice Develops Reciprocity and Cooperation Among 
Students 

3. Good Practice Uses Active Learning Techniques 

4. Good Practice Gives Prompt Feedback 

5. Good Practice Emphasizes Time on Task 

6. Good Practice Communicates High Expectations 

7. Good Practice Respects Diverse Talents and Ways of Learning 
(Chickering & Ehrman, 1987) 
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The current study addressed these principles using two key components- 
short, self-contained lessons and an online learning community. The 
short format of the lessons supports principle five by encouraging 
maximum focused effort on the task. Research on attention span and 
recall has indicated that the first five minutes of a lesson are the most 
likely to be recalled, followed by a slightly lower, but consistent level of 
recall during the next ten minutes. (Burns, 1985) Therefore, the short 
format is ideal for gaining maximum recall benefits with the shortest 
possible time investment by busy learners. Prompt feedback (principle 4) 
was provided through DLPT-style questions and answers, transcriptions, 
and translations with which the learner could immediately check 
comprehension. A variety of tools were provided in each lesson, enabling 
the linguist to tailor the lesson to fit individual learning styles and 
proficiency levels (principle 7). 

Principles 1-3, 6 and 7 were supported by a moderated listserv, 
used to establish an online learning community. The moderator actively 
contacted participants, and encouraged them to work together. Being 
part of a professional community of linguists, including native speakers, 
generated high expectations for linguists of all proficiency levels. 

Linguists were encouraged to learn actively by completing a 
comprehension task, (i.e., reading a text, listening to an authentic news 
broadcast), answering questions, then cooperating and sharing with other 
participants to verify answers or ask questions. 

Methodology 

The objectives of this study were to examine the feasibility of 
supporting Russian language maintenance by providing daily exposure to 
the language with lessons sent via email to the linguists and by forming 
an email-based learning community. Following is a description of the 
methodology used. 

Subjects 

The intent of the researchers was to contact every Russian 
linguist in the USAF to involve them in this research. However, identifying 
them and subsequently locating them and determining their email 
addresses proved much more difficult than anticipated. This effort was 
initially undertaken by the researchers, with additional help provided by 
other resources in their department. However, due to the time intensive 
nature of the task, a private contractor was hired to locate the subjects 
and their addresses, and assemble a database with the information. 
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The search entailed obtaining two databases from the Air Force 
Personnel Center, all identified linguists (those having self-identified some 
level of Russian language capability at some point in their careers), and a 
listing of those that had taken the DLPT in the previous calendar year. 
Both databases listed names; PAS Codes, which identified to which 
military personnel flight an individual was assigned; and telephone 
numbers, which were often just four digit extensions accessible locally. 
This limited information required in many cases significant effort to make 
contact, and some of the information was out of date. The primary 
element required for this study, email addresses, was not included in 
either database, and such a database does not exist in the USAF. Due 
to this required effort, the notion of including all USAF linguists was 
abandoned. Instead, a date was established, and the number of linguists 
identified by that date was the number included in the study. 

The total number of subjects participating varied due to both 
attrition and individuals requesting to be added after it had begun, and it 
varied between the two distribution lists. A chronology of the number of 
participants is in Table 1 . 

Table 1: Number of Study Participants 



Date 


Event 


Number of Participants 


Oct. 2 


Original established deadline for locating 
participants 


Announce List-290 
Discuss List-290 


Oct. 4 


Study listserve and two community lists 
established 


Announce List-290 
Discuss List-290 


Oct. 10 


Start of study/Lesson 1 


Announce List-307 (added 
24, lost 7) 

Discuss List-304 
(added 24, lost 10) 


Oct 23 


End of Study/Lesson 1 0 


Announce List-304 (added 
7, lost 10) 

Discuss List-299 
(added 7, lost 15) 



Wide variety existed in the subject pool, therefore a typical profile cannot 
be established. However, following are facts on the participants which 
provide a partial description: 

The average score of those that reported taking the DLPT was Listening 
2.18/Reading 2.32 (plusses were converted to point .5, such as 2+ 
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equaled 2.5, for calculation purposes). The number of subjects reporting 
DLPT scores was 133. The high was 3/3, and the low was 0+/0. 

Subjects were members of the active duty USAF, USAF Reserve, 
Air National Guard, US Army, US Marine Corp; US Air Force Academy 
Cadets, AF Reserve Officer Training Corp Cadets, and civilians. Objects 
were assigned to 61 different installations. 

Russian skills were learned at home, the Defense Language 
Institute, the USAF Academy, and through a wide variety and type of 
civilian courses. 

Software 



The ten lessons were developed by a contractor with basic 
guidance from the researchers. The instructional design for each section 
was consistent, with a menu on the left side of each screen that remained 
visible on every page of the lesson. A brief description of each page is in 
Table 2. (See appendix B for screen prints of the lesson pages.) The 
length of time required for completing each lesson was designed to be 
approximately 15 minutes. 

The course was developed using Microsoft Front Page. However, 
in pre-study trial runs, some subjects experienced problems opening 
pages due to Front Page extensions. As a result, the extensions were 
removed and the lessons were converted to pure hypertext markup 
language (HTML) prior to the start of the study. 

Table 2: Software Description 



Section Title 


Section Description 


Preparation 


This section began with one or two paragraphs of cultural notes, in 
English, to provide the necessary background for the content. 
Following was vocabulary, in Russian with English translation, for 
words that might be new or difficult for the subjects. The number of 
vocabulary words ranged from 12-75. 


Text/Content 


The content consisted of either a Russian language written text, a 
video/audio file, or an audio file. The text-based lessons ranged from 
3 to 6 paragraphs in length, and the audio/video based lessons were 
30 to 92 seconds in length. 


Quiz 


Quizzes were comprehension questions with both questions and 
answers in English (similar to the DLPT). Questions were offered at 
three levels, with Level 1 being the easiest and level 3 the most 
difficult. Additionally, some of the quiz sections had fill in the blank 
questions. 



7 




14 



Answers 


This page provided the correct letter responses to the quiz questions 
followed by the appropriate lines of text. 


Transcription 


For those lessons with audio or video, a complete transcript was 
provided. 


Translation 


A complete Russian to English translation was provided of the text or 
transcript. 


More Options 


This section posed a question, in English, on the subject discussed in 
the lessons intended to generate additional discussion among the 
participants. 



Software used for forming the learning community and email 
distribution was Listserv Lite, developed by LSoft Corporation. This 
software was selected due to its relatively low cost, flexibility in terms of 
moderator control of the message traffic, and familiarity to the 
researchers. The software was installed and resided on a server outside 
of the military installation where the software was developed due to 
concerns that the number of users, combined with limited bandwidth of 
pipelines (T1 lines) exiting the installation, would slow the system down. 
This was of particular concern with the lessons that contained audio or 
video content. The technician/contractor operating and maintaining this 
software was designated the list administrator. 

Subjects were provided a Cyrillic font intended to simplify 
producing and submitting Russian language postings to the discussion 
list. The font, LR_RUSKI, was chosen as it required no background 
software to run in Windows and Windows applications, and was free to 
government users. Additionally, subjects required Real Audio Player in 
order to view or listen to the audio or video clips. In order to simplify the 
process of obtaining this product, the web address where a free copy 
could be obtained was provided. However, several subjects had 
difficulties with the download and some computer managers were 
reluctant to allow installation of the software directly from the internet, 
therefore an older, free version was sent to them via email attachment by 
the course moderator upon request. This also proved problematic to 
many of the participants as firewalls and local security procedures 
precluded downloading of the attachment. 

Course Conduct 



Five days prior to the beginning of lesson distribution, all subjects 
identified were enrolled by placing them on two distribution lists. The 
Announce list was to be used for distributing the daily lessons, for 
administrative announcements, for distributing the Cyrillic font and the 
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Real Audio Player for those that requested it, and any other information 
pertinent to the entire group. Second was the Discuss list, which was to 
be used for discussion of lesson content and participant interaction. Two 
lists were used to allow subjects the option of receiving only the lessons 
without joining the learning community and receiving its inherent volume 
of email traffic. 

Subjects were notified of their placement on both lists with a 
message from the list administrator, which included a brief explanation on 
how the list operated and instructions for disenrolling. At the time of 
enrollment, the lists were unmoderated; by simply replying to a list 
generated email, the reply was distributed to everyone on the list. This 
proved untenable, as numerous subjects, having failed to read or 
understand the notion that replies would be mailed to everyone, replied to 
ask administrative questions, thereby flooding the list with messages 
intended for just the moderator. The result was 7 subjects dropping from 
both lists prior to the start of the study, and an additional 10 dropping from 
the Discuss list. The decision was then made to convert both lists to 
moderated lists; replies were sent just to the moderator, who had 
approval authority for messages prior to being distributed. This afforded 
the moderator the ability to address issues intended for individual users, 
and restrict list-wide messages to only those that applied to all. 

Daily lesson distribution began with an introductory message 
explaining procedures, technology required, discussion etiquette, other 
administrative information and lesson one (see appendix C for the entire 
message). Each lesson was identified by a lesson number (1-10) and 
given a title based on content. The emails were sent each day between 
0700 and 0730 Mountain Standard Time, and consisted of the world wide 
web address where the lessons could be accessed. Each message also 
contained administrative announcements and personal notes to the 
participants in an effort to develop cohesiveness and a sense of 
belonging. The moderator filtered replies to the list, redistributing 
messages that pertained to the content. He also sent personal emails 
back to those that were experiencing problems or wished to disenroll. On 
the day following the 10 th lesson, the survey questionnaire was mailed 
(See appendix D for the entire questionnaire). Reminders to complete 
the survey were also mailed four days and again at nine days after the 
last lesson. Following the final reminder, a total of 79 completed surveys 
were returned. 
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Results and Discussion 



Results and discussion are best separated into the following three 
categories: technology and procedural issues, language and software 
issues, and attitudinal issues. 

Technology and Procedural Issues 

As stated earlier, a primary goal of this study was to test the feasibility of 
pushing language maintenance materials and guidance to personnel 
stationed in a variety of locations worldwide on a recurring basis using 
existing technological capabilities. Generally speaking, the delivery 
system used, direct delivery of lessons via email, was a resounding 
success. 

Daily Delivery: Research on language learning strategies employed by 
adult learners indicates that the strategy most often employed by 
successful learners (i.e. those who report a high self-perception of 
language proficiency, were taking fourth or fifth year language courses, or 
systematically received high grades in college language courses) was 
frequent functional practicing-using the target language for a real 
communication task of some sort. (Bialystok, 1981 ; Oxford & Nyikos, 
1989) Frequency was achieved by receiving short lessons composed of 
authentic materials on a daily basis. One survey respondent wrote, “It 
encouraged me to study-1 didn’t always have time, but I have studied 
more in the past 10 days than I have in the past 5 years.” Another 
agreed, “After 18 years of studying Russian, this (the daily vitamin) is the 
best way to maintain.” 

Additionally, pushing language maintenance materials to students 
is an ideal way to encourage regular study because people quite often 
feel compelled to answer their email. (Kim, 2000) As one linguist wrote, 
“This is a great concept and leaves no excuses for not at least glancing at 
a Russian reading every day or two.” Functional practice was achieved 
via the listserv, where students were afforded the opportunity to practice 
the content of the lessons through further discussion. While most of this 
discussion was via transliterated Russian, it can still be classified as 
functional use of the language. 

This delivery system also solved a major problem encountered 
during the researchers’ previous study: accessing authentic materials 
from servers around the world via the Internet. In that study, participants 
were required to locate radio or television broadcasts as content for 
assignments. Because of the rapidly changing nature of the Internet and 
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unreliability of sites located within the target language country, subjects 
were frequently frustrated by difficulties accessing authentic materials. 
(Supinski et al., 1999) Additionally, a number of studies have noted the 
danger of learners becoming overwhelmed by the vastness of the Internet 
unless provided with a specific goal or task on which to concentrate 
efforts. (Frizzier, 1995; Riel & Levin, 1990; Warschauer & Whittaker, 

1997; Zhao, 1996) By having a daily lesson based entirely on authentic 
materials and located on a reliable server, the students were not hindered 
by time wasted searching for inaccessible or out of date sources or 
overwhelmed by an unfocused search for language maintenance 
materials. As one student stated, ‘There is so much available on the 
Internet that it takes me 10-15 minutes to find something appropriate, and 
usually longer as I get side-tracked with trivia along the way. I loved 
receiving it on a silver-platter in my in-box, quick and to the point, no time 
wasted finding material.” 

Worldwide Delivery: As previously discussed, the lessons were 
maintained outside the researchers’ institution due to firewalls that 
restricted access primarily for security purposes. Since several of the 
subjects used government-owned computers that could only access web 
sites with .gov or .mil suffixes, these participants opted to use their 
personal email addresses and accessed the lessons from home. Twenty 
respondents reported accessing the lessons from both home and work. 
Several deployed personnel requested the lessons be forwarded to their 
remote locations. These requests were easily honored by adding the new 
email addresses to the list, giving learners access to the program while 
on temporary duty away from their home stations. Clearly, this delivery 
method facilitates language maintenance anytime, anywhere by 
alleviating the need to carry heavy, cumbersome study materials (e.g. 
audio cassettes and players, books, dictionaries, etc.) on deployment. 
(Valentine, 1999) 

During the course of the study, each lesson was sent out at the 
beginning of the workday Mountain Standard Time. Many subjects 
expressed a desire to have each lesson waiting in their mailboxes upon 
arrival to work. Since subjects were stationed around the world in 
different time zones, this request was impossible to accommodate for all 
subjects. However, the subjects had the option of completing the lessons 
whenever convenient, so the variation in time of receipt was not 
considered a major impediment. The only disadvantage to those not 
accessing the lessons within a few hours of delivery was in lesson 
discussions. Those who posted comments or concerns about the lessons 
later than usual often did not receive responses, as the majority of 
learners had already moved on to discuss the following lesson. 
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Another consideration was whether to send each lesson in it’s 
entirety as an attachment or to send a hyperlink to a server housing the 
lessons. Because of the limited size of most subjects’ mailboxes the 
researchers chose to send only a hyperlink. Subjects were generally 
happy with this delivery system. No problems were reported downloading 
the text-based lessons and 85% of respondents reported that the 
download time for the video lessons was within reason. 

Difficulties with Video: Although no one reported problems accessing the 
web site, there were some challenges with playback of the video lessons. 
As previously discussed, the Real Audio Player was made available to 
those that did not have it at the outset of the study. With the aid of on-site 
computer help desks and the support of the Command Language 
Program Managers, the software problems were resolved for most users; 
however, twelve respondents reported still having unresolved playback 
problems at the end of the study. Such issues underscore the need for 
participants to have a “personal concierge” available to help with 
technology-related problems. (Parry & Tu, 2001) In this study, a 
“personal concierge” would have been a resource person available via 
email or telephone who could have worked with learners (or their system 
administrators) with problems playing digital video or with displaying 
Russian fonts, another significant technical difficulty encountered by 
participants. Such a resource would create a much more user-friendly 
environment by assisting in the resolution of technical issues that would 
cause frustration and impede learning by wasting time better spent on the 
lesson. 

Russian Font Issues: Since the Cyrillic font was imbedded directly in the 
HTML formatted pages, there were no problems reading Russian from 
the online lessons. However, only one participant succeeded in sending 
a message in Cyrillic via the Discuss list that was readable by other 
subscribers. Unfortunately, there are significant differences in the 
protocols used by various email software to display non-Roman fonts. 
Although the researchers tried a variety of solutions throughout the study, 
no simple, user-friendly method of displaying Cyrillic that could be applied 
uniformly for all users was discovered. Some participants solved this 
difficulty to some extent by reverting to the use of transliterated Russian. 
(For example, “ne potomu chto” for “He noTOMy mo.”) 
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Language and Software Issues 



The response from participants regarding the organization and 
effectiveness of the lessons was overwhelmingly positive. The all- 
inclusive format was found to be user-friendly and the students 
appreciated having everything they needed to complete each lesson at 
the click of a mouse. 

Virtually all students reported using all pages of the lesson most of 
the time, with the exception of the “more options” page, which was rarely 
accessed. The few who opted not to use the transcription and/or 
translation pages were typically the most advanced learners (e.g., those 
with DLPT scores of 3/3; listening/reading). 

Lesson Length: As discussed earlier, the lessons were intended to be 
approximately 15 minutes in length. This estimate proved to be fairly 
accurate with the majority of linguists spending between 10 and 24 
minutes per lesson. At the outset of the lessons, it was recommended 
that subjects not spend more than 25 minutes on each lesson. Some 
participants with lower proficiency levels did just that; they stopped 
working after an established time limit, regardless of how much of the 
lesson was completed. This most likely contributed to the low attrition 
rate as frustration was prevented. 



Figure 1 : Software Issues 



Software Issues 

(S cde: 1 = Stroncjy Dis a^ee, 2 = Disqgree, 3 = Undedded 4 = Agee, 5 = Stron^y Agee) 

Complete vvordng of questions may be found in Appendx D 

027 lessons w ere too challenging 
026 lessons were challenging enough 
Q14 format was user-friendly 
Q12 content w as diverse enough to hold interest 
09 questions were at appropriate level 
Q1 lessons were the right length 

1 2 3 4 5 
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Difficulty of Lessons and DLPT-style Quizzes: A significant challenge 
when designing materials for linguists with a wide range of skill levels (in 
this case, from 0+ to 3, including some native speakers) is determining 
the appropriate difficulty of the materials. Mindful of the primary goal of 
language maintenance (rather than development) and in order to provide 
a challenge for the majority of learners, the researchers opted for 
demanding authentic texts and audio clips originally intended for native 
speakers of Russian. To appeal to the broadest possible audience, 
DLPT-style questions at the 1 , 2, and 3 proficiency levels were used to 
adjust the task (while using the same text) to the various skill levels of 
learners. (Garza, 1996) (For descriptions of reading and listening 
proficiency levels, see appendix A). Most respondents, including several 
at lower than 2/2 skill levels, reported that the lessons were challenging 
enough, but not discouraging, due to the inclusion of transcriptions, 
translations, and vocabulary help. (See Figure 1). The majority of 
participants attempted to answer questions at all three skill levels most of 
the time. 
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Attitudinal Issues 



Because of the short length of this experiment, it would be imprudent to 
assert that the high motivational levels reported would extend over a 
longer period. However, the enthusiastic response to these lessons 
suggests a program of this type could be highly successful over the long- 
term. Virtually all participants found that lessons delivered in this manner 
would help them maintain their current language skills. Most believed 
that completing lessons of this type would lead to improvement. (See 
Figure 2) 

Figure 2: Attitudinal Issues 



Attitudinal Issues 

(Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Undecided, 4 = Agree, 5 = Strongly Agree) 
Complete wording of questions may be found in Appendix D 



Q17 daily delivery encouraged regular study 

Q1 6 five lessons per week is the right frequency 

Q7 I would enjoy receiving daily vitamin for long- 
term 

Q6 daily vitamin would help me improve skills 
Q5 daily vitamin would help me maintain skills 
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Online community: As mentioned earlier, both GOLDEN and Englishtown 
have gone to great lengths to generate online communities of learners 
with extensive access to one another and to their instructors. Such 
communities offer powerful support to distance learning by helping 
eliminate feelings of isolation that contribute to high attrition rates, 
providing feedback, fostering effective cooperative learning, and 
incorporating levels of interactivity previously attained only in the 
classroom. This experiment lasted just two weeks and used only a 
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moderated listserv to create an online community. Although elements of 
an active community began to surface, (e.g., some members began to 
acquire distinct personalities; discussions went beyond the materials 
involved in lessons, some members began communicating with one 
another off-line and even set-up several actual face-to-face meetings) 
efforts to build a successful community were clearly too limited in time 
and in scope. Consequently, it is very easy to explain why participants 
were less pleased with the Discuss listserv. 

Initially, all 290 participants were enrolled in an Announce List to 
receive hyperlinks to lessons and a Discuss List to share information 
about Russian. In spite of the initial confusion discussed earlier with 
regards to replying to a list and complaints about an onslaught of 
unimportant emails, the number of participants in the Discuss List 
increased from 290 at beginning to 299 at the end of study. The increase 
was caused by participants encouraging fellow Russian learners to take 
part in the study because of the high quality of the lessons and because 
of the sharing of valuable information related to Russian. 

Of the 299 participants enrolled, 66 used the Discuss List to ask 
questions and to share opinions. The most frequently asked technical 
questions concerned the use of Cyrillic, Real Player, or email address 
changes. Technical questions that did not directly pertain to the entire 
community should have been addressed directly to the researchers or a 
“personal concierge” (Parry & Tu, 2001) rather than the list. Therefore, 
the moderator filtered these responses and sent general technical 
guidance to the community when necessary via the message announcing 
the next lesson. 

In terms of discussing the Russian language and the lessons 
themselves, the initial comments to the Discuss List dealt with the 
accuracy of the translations (e.g. the translation “to hope” for 
“raschityvat”) or the accuracy of the transcription of the audio (‘I have to 
disagree with your interpretation of the video. I watched the video a few 
times and can clearly hear the “ne potomu chto”). Moreover, online 
personalities began to develop (e.g., a nit picker, one with too much time, 
native speakers sharing knowledge with school learners). Along with 
these personalities, “dueling duos” arose as active participants took issue 
with one another’s opinions. Fortunately (and because the list was 
moderated), none of the discussions got out of hand to become “flame 
exchanges,” as can sometimes occur due to the increased feeling of 
anonymity afforded by distance and lack of face-to-face interaction. 
(Korenman & Wyatt, 1996) Exchanges on the Discuss list conformed 
generally to accepted guidelines for positive social interactions (e.g., 
foster candor and trust, maintain a constructive tone, maintain an 
atmosphere of respect). (Preece, 2000; Palloff & Pratt, 1999) 
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An additional type of contribution appeared after one the 
researchers (herself a Russian linguist) posed a question involving 
opinion and inference (i.e., what will happen to negotiations on reducing 
the number of nuclear war heads, especially if the US continues its 
pursuit of an anti-ballistic missile system). In addition, she mentioned a 
Web site, where one could find Russian CD’s and music videos. In 
subsequent comments, other subjects began to express their opinions 
(e.g., the future of the anti-ballistic missile treaty) as well as divulge other 
useful Russian related Web sites. 

The on-going responses from subjects, enhanced by the 
pronounced effect of few comments from the researchers, indicates the 
promise a listserv has to create an online community of Russian learners. 
(See Figure 3) Factual issues and differences of opinion were readily 
addressed, concrete online personalities began to crystallize, and 
subjects expressed their opinions as linguists and professionals in the 
field. 

Figure 3: Online Community Issues 



On-Line Community 

(Scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Undecided, 4 = Agree, 5 = Strongly Agree) 
Complete wording of questions may be found in Appendix D 



Q25 course stimulated discussion of 
content 

Q19 being part of community encourages 
skills maintenance 

Q1 8 discuss list made me feel part of 
community 

Q13 too much traffic on discuss list 
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In sum, an analysis of the Discuss List contributions shows a 
listserv can be used to address technical issues (e.g., using Cyrillic, 
downloading audio and video files). Moreover, the beginnings of an 
online community of Russian learners was created in only two weeks that 
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allowed participants to discuss subtleties of translation and transcription 
as well as to express their opinions and views on issues of concern 
relating to Russia. Unfortunately, the short duration and limited scope of 
this study prevented the creation and evaluation of an online community 
similar to those evidenced by Englishtown and GOLDEN and described in 
recent texts. (Kim, 2000; Moeller, 2001 ; Preece, 2000; Vande Vrede, 
2001 ) 

Communication Outside of Discussion List: During the course of the 
study 1 1 participants made 21 personal contacts using email addresses 
provided by the listserv. On a few occasions, participants set up face-to- 
face meetings with one another, establishing an actual learning 
community similar to the meetings between GOLDEN students. Without 
the online community, these Russian linguists would have continued to 
live in the same city completely unaware of fellow Russian speakers living 
nearby. This hybrid of virtual and actual communities is proving to be 
effective in reducing the attrition rates of successful online programs. 
(Moeller, 2001) 

DL Lessons Learned and Recommendations 

Despite the short duration of the study, the following 
recommendations can be made for providing military linguists with 
technology-based distance training for language maintenance. Once 
again, the discussion will be categorized by technology and procedural 
issues, language and software issues, and motivational issues. 

T echnical/Procedural Recommendations 



Frequent functional practice with authentic language (e.g., the 
daily vitamin) has been found to be an important study habit of successful 
language learners. (Bialystok, 1981; Oxford et al., 1993) The short length 
and all-inclusive format of the vitamin lessons made frequent language 
practice for busy learners manageable, regardless of current skill levels. 
However, several changes are recommended so all linguists may access 
technology-based language maintenance materials. 

First, a database of linguists should be established and 
maintained on a long-term basis. The Air Force could assign a 
permanent email address to each accession upon entry into the military. 

In the style of a “Yahoo!” or “HotMail” account, military members would 
keep their addresses throughout the term of military service. Since the 
address would not change based on duty location, any database created 
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to service specific groups would be easily maintainable. For example, 
after a linguist establishes language skills, a permanent email address 
would be added to an email database and distribution list. Afterwards, 
the linguist would receive email from the respective distribution list. 
Information on testing, training, assignments, or related news could be 
quickly and efficiently distributed to a specific list of military members. 

Until permanent email addresses are issued, there should be one 
central list administrator (e.g., “LingNet,” the Defense Language Institute’s 
web site) where linguists can apply to be included or deleted from lists, or 
change email addresses when necessary. Contact information for this 
central database must then be widely distributed so linguists will be 
informed of the resources available to them. The researchers are 
currently working with the Defense Language Institute and the Air Force 
Institute for Advanced Distributed Learning to make additional lessons 
available to all linguists. 

Second, digital audio and video clips in future lessons should use 
Windows Media Player (WMP), the Air Force standard. Using WMP 
would eliminate problems associated with downloading Real Player 
software onto government computers. 

Third, a hyperlink to an archive page should be regularly included 
in the emails announcing daily lessons. The archive page would contain 
a compilation of all lessons presented to the distribution list to date. 
Students would then have access to one site where they could review 
past lessons, catch up on lessons missed, or receive extra practice as 
time permits and requires (e.g., immediately preceding a DLPT). 

Fourth, depending on the resources available to produce lessons, 
lesson frequency may be decreased from five to three lessons per week. 
While only 25% of respondents requested that the frequency of lessons 
be decreased to three per week, production costs for so many lessons is 
high. If the resources are available, however, a “daily vitamin” should be 
maintained, but instructions to learners should clearly state that if they 
find participating every day overwhelming, accomplishing fewer is 
certainly acceptable. This instruction should also be accompanied by the 
caveat that fewer lessons per week reduce potential benefit and raise the 
risk of not adhering to the program. 

Finally, management of such a program, to include list 
management, overseeing moderation, insuring lessons are edited for 
correctness prior to archiving for possible future use, and developing long 
range plans should be overseen by a single entity. While this entity would 
oversee the program, it need only act as a coordinator between other 
agencies or volunteers assigned specific functions and tasks. For 
example, adequately proficient military reservist linguists could author or 
maintain lessons. Program participants could also be periodically 
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solicited to develop or edit lessons. A side-benefit of such a division of 
labor would be broader awareness of the program, a sense of ownership, 
and closer touch with the needs of the learner community due to the 
cooperative approach and broader base of contributors. Volunteer 
contributors would additionally benefit by repeated use of the language 
for real communicative tasks. 

Language and Software Recommendations 

The format and length of lessons was very well received and 
should remain basically the same. The DLPT-based content 
(transportation, science, space, natural disasters, politics, etc.) was also 
well received; however, a number of participants requested some “lighter” 
topics, such as humor and anecdotes, popular music, human-interest 
stories, etc. It was impossible to cover such a broad scope of topics in 
only ten lessons, but these topics should be included as content of future 
lessons. 

Additionally, grammar tips with short exercises should be included 
on the “more options” page. One of the strategies of good language 
learners is to recognize that language is both a communicative tool, as 
well as a set of underlying rules. (Naimon, Frohlich, & Todesco, 1975) 
Linguists would be systematically reminded of the construction of the 
language to help maintain facility in productive tasks (e.g., writing or 
speaking) and help prevent the neglect of accuracy that often 
accompanies a strictly communicative approach. (Beauvois, 1992) 

Attitudinal Recommendations 



A key factor in maintaining long-term motivation in students 
involved in DL programs is fostering a strong sense of community 
between the learners and/or instructors. (Moeller, 2001 ; Parry & Tu, 

2001 ) Over the course of time, students will become comfortable in using 
the Discuss List to keep connected to the linguist community for their 
specific language; such communities should be particularly important for 
the less commonly taught languages. 

The daily vitamin program could encourage using language for 
real communicative tasks by changing the format of the “more options” 
page to further stimulate discussion. Discussion of high-interest items 
would provide incentive to view this page. For example, lessons should 
include hyperlinks to Web sites specifically targeted for the audience: 
links to Russian online newspapers and radio stations, links to useful 
language sites administered by major universities, a link to the Foreign 
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Area Officer homepage, etc. Short guidelines for specific activities using 
suggested sites should be included to provide a goal to help focus learner 
attention. (Warschauer, 1995) The list administrator’s email should be 
provided so that linguists may help keep lessons current and relevant by 
reporting defunct or out of date sites. 

Furthermore, community members should be encouraged to use 
the list for communication in the target language through discussion of 
news, lessons, websites, or other items of interest to the group. Real 
communication in the target language fosters an atmosphere where 
interest remains high and cooperative learning can flourish, allowing 
opportunities for learners to cross Vygotsky’s “zone of proximal 
development,” the gap between what someone can learn on their own 
and what they can learn in cooperation with others who are more skilled 
or experienced. (Vygotsky, 1 962) Other positive effects of cooperation 
between learners include higher achievement, more psychological 
connection (caring, support, commitment) and health, social competence, 
and higher self-esteem. (Smith, 1995) Such communication requires a 
simple solution for displaying Cyrillic. 

Learner support is vitally important to the success of any DL 
program. The instructional effectiveness of a DL program relates directly 
to the quality of support afforded to help learners interact with the subject 
matter delivered by the technology. (Moore, 2001) The high levels of 
student frustration that result from technical difficulties can be alleviated 
when help (i.e. a “personal concierge”) responds promptly, resulting in 
greater student satisfaction with the course. (Palloff & Pratt, 1999; 
Piotrow, Khan, Lozare, & Khan, 2000) 

Conclusions and Recommendations for Further Research 

Clearly, distance learning offers great potential to help solve the 
challenges of providing on-going training and education to military 
members worldwide without the expense and hardships caused by 
moving personnel from operational units. 
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Table 3: Summary of Recommendations 



Area 


Recommendation 


Technical/Procedural 


1) Establish and maintain database of linguists, including email 
addresses. 

■ Establish permanent email addresses for all military 
members. 

■ Designate a central list administrator to maintain the list 
and widely advertise contact information 




2) Create future audio/video lessons for Windows Media Player, 
the current AF standard 




3) Maintain an archive page to allow easy access to all previous 
lessons 




4) Decrease lesson frequency from 5 to 3 per week 




5) Designate a single agency to be list moderator and to 
coordinate updating of archived lessons 

■ Use Reservists to author new lessons 

■ Solicit volunteers from Discuss list to update archive 


Language/Software 


1) Include “lighter” topics regularly (humor, human-interest, pop 
music, etc.) 




2) Include grammar tips/exercises on “more options” page 


Attitudinal 


1) Include high-interest items (i.e. Russian Web sites or info on 
testing/pay) on “more options” page 




2) Encourage use of Discuss list for real communications 

■ Foster cooperative learning atmosphere 

■ Find solution to send/receive Cyrillic font via email 




3) Provide adequate technical support; a “personal concierge” 



The “daily vitamin” delivery method, combined with the support of 
professional, cooperative-learning communities, could revolutionize the 
way military education is conducted in the very near future. Suggested 
follow-on research includes a study of longer duration to test the 
development of the learning community and learner attitudes over time 
and should include data regarding before and after program skill levels. 
Another question to be addressed is the effect of the July through 
December DLPT testing cycle on learner motivation and usage of the 
foreign language maintenance program throughout the year. 

The results of this study should also be widely applicable to the 
conduct of DL in disciplines other than foreign language education. 
Studies should be conducted to determine if the “daily vitamin” format 
would be equally successful with different subject matter and different 
audience profiles. 
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Finally, a distribution list of those within DoD who are currently 
researching and working with distance education issues should be 
compiled. A learning community of distance educators sharing expertise 
and experience should be initiated via the distribution list, allowing each 
agency to learn from the others and to progress along the road to 
productive and rewarding DL experiences throughout DoD as quickly and 
efficiently as possible. (Halloran, 2001 ) 

Truly, distance learning offers a viable solution to many of the 
training challenges facing the Air Force today. With cooperation and 
creativity, these new technologies can be quickly and effectively 
employed to the benefit of both educators and learners. 
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APPENDIX A 

Description of Proficiency Levels for Reading and Listening Comprehension 

Reading Comprehension 

Level 0 (None) No practical understanding of the written language. 
Understanding is limited to occasional isolated words with essentially no ability 
to comprehend texts. 

Level 0+ (Memorized) Can recognize all the letters in the printed version of an 
alphabetic system and high frequency elements of a syllable or character system. 
Able to read some or all of the following: numbers, isolated words and phrases, 
personal and place names, street signs, office and shop designations. Examples 
of types of reading passages: weather maps, schedules, programs, menus, 
numbers, any text in which meaning is conveyed only via lexicon. 

Level 1 (Elementary) Can comprehend very simple connected written material 
in a form equivalent to usual printing or typescript. Examples of types of reading 
passages: newspaper announcements, sale ads, bulletin board information, 
invitations, tourist information. 

Level 1+ (Elementary) Sufficient comprehension to understand simple 
discourse in printed form for informative social purposes. Can guess at 
unfamiliar vocabulary if highly contextualized, but with difficulty in unfamiliar 
contexts. Examples of text types: see level 1 and level 2. 

Level 2 (Limited Working) Sufficient comprehension to read simple authentic 
written material in a form equivalent to usual printing or typescript on subjects 
within a familiar context. Examples of text types: factual descriptions, narrative 
reporting where the author is invisible or neutral, general schema, instructions, 
directions, materials addressed to less experienced native speakers. 

Level 2+ (Limited Working) Sufficient comprehension to understand most 
factual material in non-technical prose as well as some discussions on concrete 
topics related to special professional interests. Examples of text types: see level 
2 and level 3. 

Level 3 (General Professional) Able to read a variety of authentic prose 
material on unfamiliar subjects within a normal range of speed and with almost 
complete comprehension. 
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Listening Comprehension 

Level 0 (None) No practical understanding of the spoken language. 
Understanding is limited to occasional isolated words with essentially no ability 
to comprehend communication. 

Level 0+ (Memorized) Sufficient comprehension to understand a number of 
memorized utterances in areas of immediate needs. Slight increase in utterance 
length understood, but must make repeated requests for repetition and requires 
frequent long pauses between understood phrases. Understands with reasonable 
accuracy only when this involves short memorized utterances or formulae. 
Utterances understood are relatively short in length. Misunderstandings arise due 
to ignoring or inaccurately hearing sounds or word endings (both inflectional and 
non-inflectional). Distorting the original meaning, can understand only with 
difficulty even such people as teachers who are used to speaking with non-native 
speakers. Can understand best those statements where context strongly supports 
the utterance’s meaning. Gets some main ideas. 

Level 1 (Elementary) Sufficient comprehension to understand utterances about 
basic survival needs and minimum courtesy and travel requirements. Can 
understand simple questions and answers in areas of immediate need or on very 
familiar topics. Understands simple statements and very simple face-to-face 
conversations in a standard dialect. These must often be delivered more clearly 
than normal at a rate slower than normal with frequent repetitions or paraphrase 
(that is, by a native used to dealing with foreigners). Once learned, these 
sentences can be varied for similar level vocabulary and grammar and still be 
understood. In the majority of utterances, misunderstandings arise due to 
overlooked or misunderstood syntax and other grammatical clues. 

Comprehension vocabulary inadequate to understand anything but the most 
elementary needs. Strong interference from the candidate’s native language 
occurs. Little precision in the information understood owing to the tentative state 
of passive grammar and lack of vocabulary. Comprehension areas include basic 
needs such as: meals, lodging, transportation, time, and simple directions 
(including both route instructions and orders from customs officials, police 
officers, etc.). Understands main ideas. 

Level 1+ (Elementary) Sufficient comprehension to understand short 
conversations about all survival needs and limited social demands. Developing 
flexibility evident in understanding into a range of circumstances beyond 
immediate survival needs. Shows spontaneity in understanding by speed, 
although consistency of understanding uneven. Limited vocabulary range 
necessitates repetition for understanding. Understands more common time forms 
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and most question forms, some word order patterns, but miscommunication still 
occurs with more complex patterns. Cannot sustain understanding of coherent 
structures in longer utterances or in unfamiliar situations. Understanding of 
descriptions and the giving of precise information is limited. Aware of basic 
cohesive features, e.g., pronouns, verb inflections, but many are unreliably 
understood, especially if less immediate in reference. Understanding is largely 
limited to a series of short, discrete utterances. Still has to ask for utterances to 
be repeated. Some ability to understand facts. 

Level 2 (Limited Working) Sufficient comprehension to understand 
conversations on routine social demands and limited job requirements. Able to 
understand face-to-face speech in a standard dialect, delivered at a normal rate 
with some repetition and rewording, by a native speaker not used to dealing with 
foreigners, about everyday topics, common personal and family news, well- 
known current events, and routine office matters through descriptions and 
narration about current, past, and future events; can follow essential points of 
discussion or speech at an elementary level on topics in his/her special 
professional field. Only understands occasional words and phrases of statements 
made in unfavorable conditions, for example through loudspeakers outdoors. 
Understands factual content. Native language causes less interference in 
listening comprehension. Able to understand facts, i.e., the lines but not between 
or beyond the lines. 

Level 2+ (Limited Working) Sufficient comprehension to understand most 
routine social demands and most conversations on work requirements as well as 
some discussions on concrete topics related to particular interests and special 
fields of competence. Often shows remarkable ability and ease of understanding, 
but under tension or pressure may break down. Candidate may display weakness 
or deficiency due to inadequate vocabulary base or less than secure knowledge of 
grammar and syntax. Normally understands general vocabulary with some 
hesitant understanding of everyday vocabulary still evident. Can sometimes 
detect emotional overtones. Some ability to understand implications. 

Level 3 (General Professional) Able to understand the essentials of all speech in 
a standard dialect including technical discussions within a special field. Has 
effective understanding of face-to-face speech, with normal clarity and speed in a 
standard dialect, on general topics and areas of special interest; understands 
hypothesizing and supported opinions. Has a broad enough vocabulary that 
rarely has to ask for paraphrasing or explanation. Can follow accurately the 
essentials of conversations between educated native speakers, reasonably clear 
telephone calls, radio broadcasts, news stories similar to wire service reports, oral 
reports, some oral technical reports and public addresses on non-technical 
subjects; can understand without difficulty all forms of standard speech 
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concerning special professional field. Does not understand native speakers if 
they speak very quickly or use some slang or dialect. Can often detect emotional 
overtones. Can understand implications. 
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APPENDIX B 

Screen Prints of Russian Daily Language Maintenance Program 
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npecTHtic - prestige 


Answers 


HaMepeHa - is going to, intends 


jCMHTaTfe - to consider 


Translation 


cpaBHHTfe c 3EMjiBft - level to the gorund 


BBCbMa - quite 


coo6uteHite - message 


j6ecnoKoftcTBQ - concern 


More Options 


t 

ornpaBjieHHoe - sent 


'paDAenan. - to share 



iicxoAHTb - to come from 


icmpoHa - side 


paxeTHiiKH - missile troops personnel 


nepenOATOTOBKit - redeploying 


'HeAOBOJibHbifl - disgruntled 


npaBHTBJIfeCTBO - government 


AjniTejibHbxfl - lengthy 


HccjiBAOBaratH - research 


'saAepMaca - delay 


j OTCjiBHCHBaTb - to monitor 


3apnjiara - payment, salary 


noHBJieHHB - appearance 


<DEP = ^egepajifeHoe 6topo paccneflOBa- 


HAepHbift - nuclear 


hhH - Federal Bureau of Investigation 


yrpma - threat 


cBA3biBaTbca - to contact 


riepeAHCjiomrpoBaTfe - to move 


rpastiTb - to threaten, to face 


PJIC = paAno-jioKaqHOHHafl crraHUHfl - 


itcnpaBirrejibHafl kojiohhh - penal colony 


radar 


i<5CB = (JieAepanfeHafl cnyat6a 6eaonac- 


npemnojiDiKiiTejifeHo - allegedly 


hocth - Federal Security Service 


cAB«HTb - to follow, to monitor 


uiytKa - joke, prank 


rrpo6Hbifi - test 


jonacHOCTb - threat 


nycK - launch 


|ocymocTBjieHHB - implementation 


Kopafijife - ship 
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Text 



Home 

Preparation 
Text 
Quiz 
Answers 
Translation 
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3to 6bi.no KoimviapHoe npaflynpeacfleHMe: "KoiejmcKafl aubioilh PBCH naMepeHa 
cpaBHHTfe c leMJieuHBCKOJXbKO BBponeftcKHX ropoAOB 11 , Kaaajiocfe, hto coo6meHne, 
oTnpaBjieHHoe no KHrepHery, ucxoaht ot poccnftcKiix paxeTAHKOB, hbao BOJibHbix 
AJniTejibHbiMH aanepvKKaMir sapruiaTti, 

<I>EP cBaaajiocfe c KpeMJieM h xenepb wayM cTyflBHTaM TBXHHHecKoro KOJuie^ca 
ropojja Kanyra, hto k tory ot Mocicbu, rpimiT roa ncnpaBHiejifeHOfl kojiohiih. TIo 
cjioBaM <DCE, noHHTHo, hto npeAynpe^vtcmiB 6buio uiyTKofl, oflHaico onacHocTb Toro, 
MTO pOCCHftCKHB BOflCKa MOryT noftTH Ha OCymeCTRJieHHB TBpaKTOB H3-3a OTCyTCTBHfl 
aapnjiaTbi h norepii npecTtcaca, Ha 3anaAB CMmawT BBCfeMa psajifeHOfl; 3T0 
6ecnoKoflcTBo pasAejiHe? h pocciiHCKaa dopoHa. 



Ha 3 T 0 ft HBAOJTB BO BjiaflHBOCTOKB npoftABT K OHC^Bp CHUJi-H, Ha KOTOpOft 6 yAyT 
o 6 cy»flaTfeca Bonpocu nspenOATOTOBKH poccuftcKicc o^mitBpOB. Ee cncmcopaMH 
fl B-Jlfl 10 T C H 6 pHTaHCK 0 B npaBHTBJIfeCTBO II MeaatyHapOflHfexfl HHCTHTyT 
CTpaTeniHecKHX HCCJieAOBaHHft. <I>EP OTcjiBnaiBaeT noflBJieHHe ha^Phux yrpoa b 
HHT epHeTe, a ITeHTaroH nepe«ncjiOAHpoBaji rirraitTCKyio PJIC H 3 KajmtJjoptoni b 
H opBenno, npeAnojioacHTejibHo, hto 6 u cjieflHTfe 3 a npo 6 HbiMH nycxaMH paiceT c 
p o c cnftcKin xop a 6 jieft, 
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Answer the following questions based on the text (each 
question may or may not have more than one answer): 

Level 1 



1. The gist of the article is: 

a. disruptive Internet prank messages 

b. possibility of terrorist attack 

c. FBI and Pentagon 

d. Federal Security Service 

2. The message was sent by: 

a. personnel of Kozelsk division 

b. FBI 

c. students 

d. officers 

3. Who is sponsoring the upcoming conference? 

a. the British Government 

b. the International Institute for Strategic Studies 

c. the Kremlin 

d. both a. and b. 
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Level 1 



1 - B : 

2 - C : 

<3>EP CBH 3 ajioci, c KpeivuieM n Tenept jcbvm cryneHtaM TexHtreecKoro Kojuiegaca 
ropofla Kajiyra, hto k lory ot Mockbbi, rpo3HT ron ncnpaBiuejiBHofl kojiohhh . 

3-D: 

... Ee cnoHcopaMH hbjihiotch SpHTaHCKoe np aB ht e jh> c t b o h MeagtyHapoflHMH 
HHCTnryr crparennecKHX nccneflOBaioifl. 



Level 2 

1" D : j 

Kaaajiocb, uto coofimemie , orrrpanjiem:oe no MniepHeTy, hcxozpit ot pocchAckidc I 
paKeruHKOB ! 



2 - B : 

Ha 3Toil Heflene bo BjiaflHBOCTOice npofmeT KOH^epenmia, Ha KOTopoii 6ygyT 
oficyac- flaTbca Bonpocu nepenoOTQTOBKH pocchhckhx odmuepoB . 

3-: 

... gsyM crvneHTaM TexuiriecKoro tcoJine^Ka ropona Kajiyra , hto k lory ot 
Mockbu, rpoiHT rofl ncnpaBHTejn>Hoft kojiohiul 
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It was the nightmare warning: "The Kazelsk Division of the Strategic Rocket Forces 
intends to level to the ground a number of cities in Europe." Sent by Internet, the 
message claimed to be from Russian missile troops disgruntled after going for a long time 
without pay. 

The FBI contacted the Kremlin and two students from a technical college in Kaluga, south 
of Moscow, now face a year in a penal colony. The warning was meant as a prank, 
Russia's Federal Security Service said, but the risk of Russian troops resorting to 
terrorism over their loss of pay and prestige is seen in the West as very real and is of 
concern in Russia too. 



The British Government and the International Institute for Strategic Studies are 
sponsoring a conference in Vladivostok this week on redeploying Russian officers. The 
FBI is monitoring nuclear threats on the Internet and the Pentagon has moved a giant 
radar dish from California to Norway, allegedly to monitor test missile launches from 
Russian ships. 






£ile gdit Jfi aw Favorites Tools fcfelp 



Home 

Preparation 
Text 
Quiz 
Answers 
Translation 
More Options 



More Options 

As a part of military, what do you think about such risk? What do you think 
should and can be done to prevent it from happening? 
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APPENDIX C 

Introductory Letter and Instructions 
Welcome Russian Linguists! 

First I wish to apologize for the extraneous emails you received last week. This 
will not happen again. 

This is the first of 10 lessons you will receive in a study the US Air Force 
Academy Department of Foreign Languages is conducting on the use of daily 
Russian lessons delivered via your email. Below is the web address where the 
lesson is located. Once you have read the administrative notes below, simply 
click on the address. (Note: Please keep this message for reference purposes 
either electronically or in hard copy. There are also some notes within the first 
lesson that explain how to use it.) 

Administrative Notes: 

1. This is a voluntary research study, but we ask that you stay with this program 
for the entire 10 work days. Please keep our goals in mind: we want to determine 
how good the lessons are, how you like receiving them daily, whether you think 
that they will help you maintain Russian proficiency, and whether you enjoy 
being part of a learning community. All we ask is that you do some of the 
lessons, and after lesson 10 you complete our evaluation survey. We have 
already been funded to provide many more of these lessons, and we want your 
feedback to make them as useful as possible. 

2. The lessons will be in your mailbox each day for use when you are ready for 
them. We recommend you set aside the 15 (or so) minutes each day it takes to 
complete the lesson and develop a daily routine. This routine should include 
closing your door or some other method of insuring the 15 minutes are 
uninterrupted. The lessons may be a bit difficult for some of you (designed for 
the 2/2 level and higher); if so, decide how long you wish to work (15-20 
minutes) then quit. Remember that you can save the emails and come back to 
them another time. 

3. In addition to the lessons, you are part of a “listserv.” The purpose is to tie 
those of us that wish to maintain our Russian into a learning community. The 
OPTIONS section of each lesson is designed to encourage more discussion of the 
topic at hand. This may be done in English or Russian, but, since this is intended 
to improve your Russian skills, Russian is preferred. To facilitate this, if you are 
using Microsoft Outlook, attached below is a font, LR_Ruski, that you can load 
and use. Simply save the file, then open start, control panel, fonts, and follow 
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instructions to install a new font. If you have problems doing this, call or email, 
and we’ll work to solve the problem. 

(LR Ruskifont file) 

a. There are actually two lists. This is the “Announce” list and is the list 
that the sends the email that contains the lessons. 

DAILY-RUSSIAN-ANNOUNCE- 

L@LANGUAGES.DATAWEST.NET 



This list will also be used to relay important information about the 
course, technical issues, and to distribute the survey. 

b. The second list is the “discuss” list. 

DAILY-RUSSIAN-DISCUSS-L@LANGUAGES.DATAWEST.NET 

This list is for Russian or English language discussion about a lesson and 
more informal topics. These emails will automatically be sent to the all 
the other members (approximately 300) of this group. You can do this 
by replying to the email which contains the lesson. The email will be 
held for review. These announcement emails are viewed by me before 
they go to everyone on the list; I will correct any Russian language 
content before I send to everyone else on the list-we don’t want our 
language learners getting incorrect Russian! 

c. You have been subscribed to both lists. If you wish to receive the 
email lessons but not the more informal discussion emails, unsubscribe 
from 

DAILY-RUSSIAN-DISCUSS-L@LANGUAGES.DATAWEST.NET 
Instructions on how to unsubscribe from a list are included below. 

d. If you do send messages to either list, please adhere to “netiquette” as 
follows: 

• Include a subject line for your postings, and ’’sign” your messages. 

• Think before you post, and please, no flaming others publicly. 

e. We developed two lists for those who may wish to receive the lessons, 
but do not want the additional email traffic of discussions. If you must 
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take yourself off one or both lists, please follow the instructions you 
received in the email which signed you on. There are also instructions 
on how to unsubscribe at the bottom of every email sent from both lists. 
We ask, however, that you send me (Stanlev.Supinski@usafa.af.mil) a 
separate email telling us why for the purposes of our study. 

f. In addition to the group “chat” you may also wish to connect with 
another person on the list “off line” as a pen pal. A listing of who is on 
the list and their email addresses may be retrieved by sending 

http://www.usafa.af.mil/dff/distro/list.htm 

g. If you feel you have missed a message, an archive of previous 
messages can be retrieved by email from the Listserv. Send an email to 
listserv@languages.datawest.net . in the body of the message include the 
single word “help” (without the quotes.) You will receive instructions on 
how to retrieve archived messages. An alternative method for reading 
previous messages is via a web interface at 
http://languages.datawest.net/scripts/wa.exe 

4. The audio and video content in the lessons requires the use of real audio. If 
you do not have it, the program may be downloaded free from their website at 

Real Audio Download Link 



if that link does not work try: 
http://www.realaudio.com 

5. Thanks in advance for your participation and for helping us develop a system 
to maintain your language skills. Please contact either of us if you have any 
problems or concerns. 

Lt Col Stan Supinski Capt Sue Valentine 

DSN 333-8680 DSN 333-3820 

Stanle v.Supinski @ usafa.af.mi 1 Susan.valentine@usafa.af.mil 

6. If you are having any specific technical questions about using the Listserv or 
the Listserv website please email: listadmin@datawest.net 

Lesson 1 is at (just click on the address below): 

http://www.usafa.af.mil/dff/distro/lessonl/index.htm 
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APPENDIX D 

End of Course Survey 

For the questions with a rating scale, please click on the underline 
corresponding to your chosen response and type "x". Please add any 
additional comments that may help clarify or explain your rating below the 
box. 

1. The lessons were about the right length. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

2. Please tell us approximately how long you spent on each lesson on 
average. 

SCALE: Shorter time , 10-15 min. , 15-19 min. , 20-24 min. , 

25-30 min. , longer time . 

3. Of the 10 lessons sent out, I worked on . (How many? 1-10.) 

4. I worked on the lessons the same day as I received them. 

SCALE: Never , Rarely , Sometimes , Usually , Always . 

(If you said "never," "rarely," or "sometimes" please tell us whether it was 
due to time constraints, technical problems, etc...) . 

5. I feel a daily Russian "vitamin," such as these 10 sample lessons, would 
definitely help me maintain my skills. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

6. I feel a daily Russian "vitamin," such as these 10 sample lessons, would 
definitely help me improve my skills. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

7. I would enjoy receiving a Russian lesson daily in my email on a 
long-term basis. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

8. I would never recommend signing up for this distribution list to my 
friends or colleagues. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 
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Strongly Agree . 

(Please tell us why or why not. Also, please tell us if you did recommend 
someone else sign up & if so, how many people). 

9. The questions, for the most part, were appropriate for my language skill 
level. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

10. My most recent DLPT scores were: Reading; Listening; (or 

mark an x here: to indicate you haven’t taken the DLPT recently.) 

11. I usually completed the questions for level: (check all that apply) 

SCALE: Level 1 , Level 2 , Level 3 . 

12. The content of the lessons was diverse enough to keep me interested. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

(Please suggest topics for which you’d like to see lessons in the 
future: ) 

13. I felt there was too much message traffic on the "discuss" list. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

14. I felt the lesson format was user-friendly. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

(Please tell us what you found most/least useful and give us any 
suggestions about what else you would include.) 

15. 1 normally used the following links in the lesson: (check all that 
apply) 

SCALE: Preparation , Quiz , Answers , Transcription , 

Translation , More Options . 

16. Receiving short lessons 5 days per week is about the right amount for 
me to be able to maintain my Russian skills. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 
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(If you disagreed, please tell us how often you would rather receive 
them). 

17. Having the lessons sent to me daily encouraged me to study regularly. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

18. The "discuss" list helped me feel more a part of the Russian linguist 
"community." 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

(Please tell us whether or not you contacted any members of the list 

directly, as opposed to responding to the entire list) yes; no. How 

often? 

19. Being a part of a "community" of linguists encourages me to maintain or 
improve my Russian skills. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

(Please tell us what would encourage you, if this isn’t it). 

20. I’m using the following operating system: 

SCALE: Windows 95 , Windows 98 , Windows 2000 , Other . 

(If "other, "please tell us which one). 

21. I usually completed the lessons at the following location: 

SCALE: At home , At work , 1/2 at home-1/2 at work , Other . 

22. The video lessons take too long to download. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

23. My favorite type of lesson was: 

SCALE: Reading , Audio/Video , Audio only . 

24. The skill I feel I’ve most improved as a result of doing these lessons 
is: 

Listening , Reading , Writing , Speaking , The lessons didn’t 

help . 
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25. The course effectively stimulated student discussion of the content. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

26. The lessons were sufficiently challenging to present growth 
opportunities. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

27. The lessons were so difficult that they discouraged me from continuing 
to use them. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

28. I would like to participate in an in-country immersion program to improve 
my language skills. 

SCALE: Strongly Disagree , Disagree , Undecided , Agree , 

Strongly Agree . 

Free-response questions: 

Directions: There is no rating scale for the following questions. Please 
respond freely. 

1. Did you have any technical problems? (i.e. problems reading the 
Cyrillic, problems running the videos, problems typing in Cyrillic, etc.). 

If yes, please describe: 

2. If you had technical problems, who solved them and approximately how 
much time did it require? 

3. Were your local computer folks helpful in solving any problems you had. 
(yes/ no/ not applicable). If not, please explain the situation. 

4. What is one thing you would change about the lesson format? 

5. What is one thing you would change about the lesson content? 

6. What else would you like to tell us? 



You are subscribed to DAILY -RUSSIAN-ANNOUNCE-L. For subscription 
info and archives see http://languages.datawest.net/scripts/wa.exe . 
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To unsubscribe send an email to 

mailto:LISTSERV@LANGUAGES.DATAWEST.NET . In the body include: 
UNSUBSCRIBE DAILY -RUSSIAN-ANNOUNCE-L (email addr) 
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